Streaming media, video, audio, animation, or otherwise, has become very popular as network connection bandwidth and quality has increased. When a particular selection of media is stored on a server, that selection can be streamed to one or more requesting devices on demand. Moreover, to account for network issues that may inhibit the media stream, the server can transfer the media stream faster than it can be presented at the receiving device. This faster transfer allows the receiving device to store the media locally for presentation in a process commonly called buffering. If a media server or network issue occurs that cuts off or slows the media stream, the buffered media is unaffected, which allows for unhampered presentation of the media and provides extra time for the server or network to recover.
While the above buffering process works for media that exists on a server and can be sent ahead of time to a device, in the case of live or real-time media streams, that luxury does not exist. In particular, during live content streaming, the delay caused by storing media in a buffer may be detrimental to the live presentation experience. Likewise, with real-time communications, excess delay in presenting other parties on a communication harms the flow of the communication session. Accordingly, issues that arise when transferring live and real-time communication streams are not able to be mitigated using a traditional buffering scheme.
Overview
Embodiments disclosed herein provide systems, methods, and computer readable media that perform adaptive management of a media buffer. In a particular embodiment, a method provides receiving a plurality of packets representing a media stream into the adaptive media buffer, wherein the media stream comprises a sequence of media segments structured into a plurality of segment layers, and detecting loss of one or more packets of the plurality of packets, wherein the one or more packets represent a media segment of the plurality of media segments. The method further provides estimating a round trip delay for packets between the adaptive media buffer and a sender of the media stream and determining a current depth of the media segments within the adaptive media buffer. In response to detecting the loss of the one or more packets, the method provides discarding one or more media segments of the media segments within the adaptive media buffer based on the round trip delay, the current depth, and a layer of the plurality of segment layers into which each of the one or more media segments is included.
In some embodiments, the method further provides, in response to detecting the loss of the one or more packets, determining whether to request retransmission of the one or more packets based on the round trip delay and the current depth.
In some embodiments, upon determining to request retransmission, the method provides requesting the one or more packets from the sender and holding frames starting with the media segment until the one or more packets are received from the sender, and, upon determining not to request retransmission, the method provides discarding the media segment and other media segments that refer to the media segment from the adaptive media buffer.
In some embodiments, upon receiving the one or more packets from the sender, the method provides passing the media segment to a decoder and passing remaining media segments to a decoder at a rate that speeds up presentation of the media stream until buffer delay returns to a threshold delay value.
In some embodiments, the plurality of segment layers include a base layer and a plurality of enhanced layers that each refer either to a media segment of the base layer or to a media segment of a lower enhanced layer of the plurality of enhanced layers.
In some embodiments, a media segment included in a higher level enhanced layer is more likely to be discarded than a media segment in either a lower level enhanced layer or a base layer.
In some embodiments, discarding the one or more media segments is further based on content priority indicators associated with the media segments.
In some embodiments, estimating the round trip delay comprises measuring a depth of the media segments within the buffer during one or more packet retransmission events and determining the round trip delay based on the depths of the media segments within the buffer during the one or more packet retransmission events.
Another embodiment provides a non-transitory computer readable medium having instructions stored thereon. When executed by a processor system, the instructions direct the processor system to perform a method of managing an adaptive media buffer. The method includes receiving a plurality of packets representing a media stream into the adaptive media buffer, wherein the media stream comprises a sequence of media segments structured into a plurality of segment layers, and detecting loss of one or more packets of the plurality of packets, wherein the one or more packets represent a media segment of the plurality of media segments. The method further includes estimating a round trip delay for packets between the adaptive media buffer and a sender of the media stream and determining a current depth of the media segments within the adaptive media buffer. In response to detecting the loss of the one or more packets, the method provides discarding one or more media segments of the media segments within the adaptive media buffer based on the round trip delay, the current depth, and a layer of the plurality of segment layers into which each of the one or more media segments is included.
Yet another embodiment provides an adaptive media buffer management system comprising a communication interface, a processing system, and a memory system. The communication interface is configured to receive a plurality of packets representing a media stream into the adaptive media buffer, wherein the media stream comprises a sequence of media segments structured into a plurality of segment layers. The processing system is configured to detect loss of one or more packets of the plurality of packets, wherein the one or more packets represent a media segment of the plurality of media segments, estimate a round trip delay for packets between the adaptive media buffer and a sender of the media stream, determine a current depth of the media segments within the adaptive media buffer, and, in response to detecting the loss of the one or more packets, discarding one or more media segments of the media segments within the adaptive media buffer based on the round trip delay, the current depth, and a layer of the plurality of segment layers into which each of the one or more media segments is included. The memory system is configured to store the adaptive media buffer.